Papers with Fine-tuning language models
CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in instruction-tuning datasets focus on specific tasks like mathematical or logical reasoning. |
| Approach: | They propose to use synthetic dialogues to help language models remain focused on the subject at hand during task-oriented interactions. |
| Outcome: | The proposed dataset improves language models' ability to maintain topical coherence compared to general-purpose instruction-tuned LLMs like gpt-4-turbo and Mixtral-Instruct. |
NSF-SciFy: Mining the NSF Awards Database for Scientific Claims (2026.acl-long)
Copied to clipboard
| Challenge: | NSF-SciFy contains 2.8 million claims from 400,000 abstracts spanning all science and mathematics disciplines. |
| Approach: | They propose to use a dataset to extract scientific claims from National Science Foundation award abstracts and to use it to refine language models. |
| Outcome: | The proposed method improves non-technical abstract generation, claim extraction, and investigation proposal extraction tasks while maintaining high precision and lower recall. |